<html>
<head>
	<title>Wiki: All Combinations</title>
<head>
<body bgcolor=#FFFFFF link=#d06040 vlink=#806040>
	<h1>All Combinations</h1>
	<wiki>This is the second of a suite of fixtures that make up the AllPairsExample. Like with AllFiles, we expand wildcard characters in path specifications to get lists of tests. But this time we run all combinations of the tests. This is another warmup for AllPairs testing. 
<p><p><hr>
<p>A project can expect to have dozens or hundreds of files testing specific features in a stand-alone way. We may wonder if there are any interactions between features that will cause any of these tests to fail. AllCombinations explores this. It is a fixture that enumerates cases by drawing one file at a time from each of a number of lists; running each in turn, and then repeating the drawing and running for all possible combinations.
<p>For the purpose of illustration we've created a number of files that contain small tests of the CalculatorExample. We've categorized the files as <em>magnitudes, signs</em> and <em>functions,</em>  which enter values, changes signs, and compute functions, respectivly.
<p><UL>
<li> <a href="Release/Documents/AllPairs/">http:Release/Documents/AllPairs/</a>
<p></UL>
This table runs all possible sequences of magnitude, sign and function. Try it with <a href="run.cgi">http:run.cgi</a>.
<p><table BORDER CELLSPACING=0 CELLPADDING=3>
<tr><td ColSpan=2> eg.AllCombinations</td></tr>
<tr><td> Documents/AllPairs/magnitude/*.html </td><td>&nbsp;</td></tr>
<tr><td> Documents/AllPairs/sign/*.html </td><td>&nbsp;</td></tr>
<tr><td> Documents/AllPairs/function/*.html </td><td>&nbsp;</td></tr>
</table>
<p>The number of cases will as large as the product of the size of each list of tests. The searches in the table find four magnitues, three signs and two functions.
<p><UL>
<li> 4 magnitudes  X  3 signs  X  2 functions  =  24 cases
<p></UL>
Any file of any case can fail. We save the results from failed tests only. A single file can fail different ways in different cases. Look for differences in the counts or examine each FileFootnote. 
<p>If the tests are intended to interact, as ours are here, then some care is required to be sure that checks in the tests are independent of this interaction. No checks at all my be suitable if we are only testing that there are no exceptions thrown. Or the simplest checks may be adaquate so long as features are tested well in isolation. It may be convenient to run a simulation, a so called <em>oracale</em>, along side the application to generate expected results. 
<p><hr>
<p><strong>Implementation</strong>
<p>The fixture inherits path expansion from AllFiles, but captures these by overrinding <em>doRows(Parse, List)</em> and saving the list of files in a list of lists. Then, at the end of <em>doTable</em>, the fixture gets to work computing and running combinations.
<p>See source.
<p><UL>
<li> <a href="Release/Source/eg/AllCombinations.java">http:Release/Source/eg/AllCombinations.java</a>
<p></UL>
We had a little trouble setting up the test data for this example so we wrote this little <em>Trigonometry Refresher</em>.
<p><table BORDER CELLSPACING=0 CELLPADDING=3>
<tr><td ColSpan=3> eg.ArithmeticColumnFixture</td></tr>
<tr><td> x </td><td> sin() </td><td> cos() </td></tr>
<tr><td> 0     </td><td> 0.0000 </td><td> 1.0000 </td></tr>
<tr><td> 30   </td><td> 0.5000 </td><td> 0.8660 </td></tr>
<tr><td> 60   </td><td> 0.8660 </td><td> 0.5000 </td></tr>
<tr><td> 90   </td><td> 1.0000 </td><td> 0.0000 </td></tr>
<tr><td> 120 </td><td> 0.8660 </td><td> -0.5000 </td></tr>
<tr><td> 150 </td><td> 0.5000 </td><td> -.8660 </td></tr>
<tr><td> 180 </td><td> 0.0000 </td><td> -1.0000 </td></tr>
</table>
<p><hr>
<p><p><table BORDER CELLSPACING=0 CELLPADDING=3>
<tr><td ColSpan=2> eg.Summary </td></tr>
</table>
<p></wiki>
<hr>
	Last edited November 4, 2002
</body>
</html>

